Model Selection in Reinforcement Learning with General Function Approximations

We consider model selection for classic Reinforcement Learning (RL) environments – Multi Armed Bandits (MABs) and Markov Decision Processes (MDPs) under general function approximations. In the framework, we do not know classes, denoted by $$\mathcal {F}$$ {M}$$ , where true models reward generating MABs transition kernel MDPs lie, respectively. Instead, are given M nested (hypothesis) classes such that contained in at-least one class. this paper, propose analyze efficient algorithms MDPs, adapt to smallest class (among classes) containing underlying model. Under a separability assumption on hypothesis show cumulative regret of our adaptive match an oracle which knows correct (i.e., ) priori. Furthermore, both settings, cost is additive term having weak (logarithmic) dependence learning horizon T.